Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental gRPC transport plugin #16534

Merged
merged 19 commits into from
Dec 31, 2024
Merged

Conversation

finnegancarroll
Copy link
Contributor

@finnegancarroll finnegancarroll commented Oct 31, 2024

Description

Auxiliary transports

Provides a framework for implementing pluggable auxiliary transports through NetworkPlugin.
Auxiliary transports run in parallel to existing http & native transport servers.
None or many Auxiliary transports may be provided and can be configured with the following settings:

aux.transport.types enables all listed transports which are installed.
aux.transport.<type>.ports configures the port range for the specified transport.

In the absence of aux.transport.<type>.ports the auxiliary transport falls back to a default port range (9400-9500).
If aux.transport.types is set and the corresponding transport plugin is not installed an exception is thrown.

transport-grpc plugin

A gRPC server plugin is implemented as an initial auxiliary transport.
This gRPC server includes only default list and Health/Check services.

To enable the gRPC plugin include the following in run.gradle.

setting 'aux.transport.types', '[experimental-transport-grpc]'
setting 'aux.transport.experimental-transport-grpc.ports', '9400-9500' //optional

Then run OpenSearch with the plugin installed.

./gradlew run -PinstalledPlugins="['transport-grpc']"

grpcurl commands for querying default services.

grpcurl -plaintext localhost:9400 list
grpcurl -plaintext localhost:9400 grpc.health.v1.Health/Check

Related Issues

RFCs detailing generation of protobuf files from api-specification - 1 2.

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for b5fddb1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 24a40ad: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll finnegancarroll mentioned this pull request Nov 4, 2024
1 task
Copy link
Contributor

github-actions bot commented Nov 7, 2024

❌ Gradle check result for 3bcd5bb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Nov 7, 2024

❌ Gradle check result for b2a1078: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Nov 8, 2024

❌ Gradle check result for ee85066: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for eda1c04: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 5038cf3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 1e5d354: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 00d4eb1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 6151622: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for b8d88d2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for b404cee: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 41e70eb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for e985043: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for bbabecd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 853b206: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for da16c66: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll finnegancarroll changed the base branch from main to feature/grpc November 23, 2024 00:17
@finnegancarroll finnegancarroll marked this pull request as ready for review November 23, 2024 00:24
Signed-off-by: Finn Carroll <[email protected]>
The motivation of this change is to allow for multiple configured
auxiliary transports. As such newAuxTransports() now returns a list
of enabled transports and each AuxTransport implements it's own
'aux.transport.type' and 'aux.transport.<type>.ports' setting. Since
Security.java initializes previous to Node.java during bootstrap
socket binding permissions are granted based on
'aux.transport.<type>.ports' for each enabled 'aux.transport.type',
falling back to a default if no ports are specified.

Signed-off-by: Finn Carroll <[email protected]>
getAuxServerTransportSupplierList -> getAuxServerTransportList
aux.transport.type -> aux.transport.types

Signed-off-by: Finn Carroll <[email protected]>
Signed-off-by: Finn Carroll <[email protected]>
@andrross andrross added the backport 2.x Backport to 2.x branch label Dec 30, 2024
Copy link
Contributor

❌ Gradle check result for ecd998e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

✅ Gradle check result for 6fc1453: SUCCESS

@andrross andrross merged commit 7a0e8fb into opensearch-project:main Dec 31, 2024
39 of 40 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-16534-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7a0e8fbed7116173e11f52236c347affd4e22a16
# Push it to GitHub
git push --set-upstream origin backport/backport-16534-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-16534-to-2.x.

@andrross
Copy link
Member

@finnegancarroll Can you create a manual backport for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport-failed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants